Shared Hierarchical Aggregation for Monitoring Distributed Streams

Widely dispersed monitoring networks generate huge data volumes that are naturally organized via hierarchical aggregation. In a system that manages such data, applications pose periodic aggregate queries. In this paper we show how to efficiently process multiple periodic aggregate queries in a hierarchy. First, we use a novel query rewrite that optimally executes individual queries. Next, we show how to combine the rewritten queries to share computation and communication resources. Finally, we identify a challenge in shared aggregation across a heterogenous hierarchy, namely that push-down reduces sharing and pull-up increases communication. We then propose a “partial push-down” technique that permits effective sharing without increasing communication costs.